Physics 



Thermodynamic formalism and large deviation functions in 
continuous time Markov dynamics 

Vivien Lecomte b , Cecile Appert-Rolland a , , Frederic van Wijland b 

a Laboratoire de Physique Theorique (CNRS UMR8627) , Bdtiment 210, Universite Paris-Sud, 91405 Orsay cedex, France 
b Laboratoire Matiere et Systemes Complexes, (CNRS UMR7057), Universite Denis Diderot (Paris VII), 10 rue Alice Domon 

et Leonie Duquet, 75205 Paris cedex 13, France 

Received *****; accepted after revision H — h + + + 



Abstract 

The thermodynamic formalism, which was first developed for dynamical systems and then applied to discrete 
Markov processes, turns out to be well suited for continuous time Markov processes as well, provided the definitions 
are interpreted in an appropriate way. Besides, it can be reformulated in terms of the generating function of an 
observable, and then extended to other observables. In particular, the simple observable K giving the number 
of events occurring over a given time interval turns out to contain already the signature of dynamical phase 
transitions. 

For mean-field models in equilibrium, and in the limit of large systems, the formalism is rather simple to apply 
and shows how thermodynamic phase transitions may modify the dynamical properties of the systems. This is 
exemplified with the q-state mean-field Potts model, for which the Ising limit q — 2 is found to be qualitatively 
different from the other cases. 

Resume 

Formalisme thermodynamique et grandes deviations dans les systemes a dynamique markovienne. 

Le formalisme thermodynamique, qui a d'abord ete developpe dans le cadre des systemes dynamiques puis 
applique aux processus de Markov, s'avere egalement pertinent pour les dynamiques de Markov en temps continu, 
a condition toutefois d'interpreter les definitions en jeu de facon appropriee. Ce formalisme peut etre reformule en 
termes de fonction generatrice d'une observable, puis etendu a d'autres observables. En particulier, l'observable 
K donnant le nombre d'evenements ayant lieu dans un intervalle de temps donne, bien que tres simple, contient 
deja la signature de transitions de phases dynamiques. 

Pour les modeles de champ moyen a l'equilibre, et dans la limite des grands systemes, le formalisme peut s'ap- 
pliquer simplement et montre comment les transitions de phase thermodynamiques peuvent affecter les proprietes 
dynamiques de ces systemes. Cela est illustre sur le cas du modele de Potts en champ moyen, et il s'avere que le 
cas d'Ising differe qualitativement des autres cas. 
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1. Introduction 

The theory of chaos has first been developed in the frame of dynamical systems. In particular the well-known 
Lyapunov exponents were then defined, which characterize the exponential divergence between initially closeby 
trajectories in phase space. Lyapunov exponents characterize individual directions, and if at least one of these 
exponents is positive, the system is said to be chaotic. The Kolmogorov-Sinai entropy provides a more global 
measurement of chaos. This can directly be seen from Pesin's theorem, which relates the KS entropy to the sum of 
positive Lyapunov exponents: has = X^a >o ^ ^ or a c l° se< A system. In practice, in order to obtain the /iks entropy, 
one has to partition phase space. A trajectory in phase space becomes, in this coarse grained description, a sequence 
of cells which are visited by the actual trajectory. By doing so, the deterministic dynamics can be turned into 
a stochastic one. The KS entropy is then defined as facs = — limt^oo | ^ hiatories Prob{history} In Prob{history}. 
Then, in order to mimic the formalism that is used for equilibrium systems, a so-called thermodynamic formalism 
was introduced [1] , in which various quantities - and in particular the KS entropy - are derived from a partition 
function 

Z(s,t)= (ProMhistory}) 1 - 8 (I) 

histories from — >t 

This function is called dynamical partition function, as the sum is not taken over configurations but over histories. 
We have replaced the usual exponent (3 by 1 — s for reasons that will become obvious later. Anyhow, this exponent 
has nothing to do with a temperature. It is just a parameter that allows to scan the distribution Prob{history}. In 
the same way as a free energy is defined for equilibrium systems, one introduces the so-called topological pressure, 
or Ruelle pressure, defined by 

rj){s) = lim - \nZ{s,t) (2) 

t^oc t 

where the thermodynamic limit is replaced by a long time limit. The KS entropy is immediately obtained from 
ip{s) via the relation has = ^'(0)- Other quantities that can be obtained from tp are for example the topological 
entropy, h top = i>{l), which counts the number of trajectories within a certain time interval, or the escape rate 
(for open systems) which is equal to 7 = — V>(0). 

As the deterministic dynamics had to be turned into a stochastic dynamics in order to apply this formalism, the 
latter is of course particularly well-suited for systems which are stochastic from the beginning, as Markov processes 
are. The formalism was successfully applied to a number of Markov processes in discrete time, and in this frame, it 
was shown that /iks could be efficiently calculated via the formulation /iks = ~ 7(Xe' ~ * O mTO (C — > 0) s t 
where w(C — > C) is the probability to jump from a configuration C to a configuration C , and r is the time step 
of the process [2] . 

If one tries to extend the calculations of /iks to continuous time by letting the time step r tend to zero, the 
probability w(C — > C') (which is proportional to r) also tends to zero and /iks diverges. In the same way, a finite 
continuous time limit for the thermodynamic formalism cannot be obtained by taking r — > 0. 

A first remark is that this divergence of /iks does reflect the fact that the dynamical complexity of the trajectories 
is infinite when continuous time dynamics is considered, since one needs an infinite amount of information to 
describe the continuum of time intervals between configuration changes (see Sec. 2). In that sense, the divergence 
is meaningful. On the other hand, this divergence indicates that the glasses one is wearing to look at Markov 
processes (discrete time) may not be the most appropriate ones if one is interested in Markov dynamics (i.e. with 
continuous time). 

In this paper, we shall summarize which glasses should be used, i.e. how the thermodynamic formalism should 
be applied in the case of Markov dynamics in order to get finite quantities [5,6]. The formalism will be rephrased 
in terms of large deviations of observables - or more precisely in terms of cumulant generating functions, which 
are the Legendre transforms of large deviation functions. This allows to cast the thermodynamic formalism into 
a more general frame including the much-discussed fluctuation theorem. 

The computation of the topological pressure ip(s) for physically relevant models is in general a difficult task. 
Here, we shall show that in the special case of mean-field models at equilibrium, the dominant term of ^(s) in 
the limit of large systems can be obtained quite simply from the minimization of a given functional. As a result, 
one obtains a description of the dynamical properties of the aforementionned equilibrium systems. This will be 
illustrated on the example of the Potts model. 
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2. The thermodynamic formalism for continuous time Markov dynamics 



We shall start with some obvious remarks on the origin of stochasticity of the trajectories. In order to define a 
trajectory, one has to determine: 

- which configurations will be visited: this is the configurational part of the trajectory, and it is just a sequence 
of configurations Co —>...—> Ck where successive configurations are different. Once we know that the system 
will jump from configuration C to another one, the probability to chose a specific configuration C' is W ^(c) C ' 
where r(C) is the rate of escape from configuration C: 

r(C) =J2W(C^ C). (3) 
The probability of the configurational part of the trajectory thus reads: 

Prob{history} = \[ W ( C ^ C ^) (4) 

n = v ' 

- when shall the system jump from one configuration to the next one: 



ti 

Co => 



t2 



t K t 

=»• C K 



(5) 



The probability density for not leaving the configuration C n -i during the time interval t n — tn-i and for changing 
configuration at time t n is r(C„_i)e'* n_tn - 1 ' T '^ Cn - 1 '. For the last time interval, one must only ensure that the 
system does not leave configuration Ck during the time interval t — tK, and the associated probability density 

i s e (t-t K )r(C K )_ 

The key point when applying the definition for the dynamical partition function is that only the probability of 
the configurational part of the trajectory should be raised to the power 1 — s, i.e. 



Z(s,t\C ,t ) = J2 Yl / dtl r(Co)e- (tl " 



to)r(Co) 



/ dt k r 



(Cfc-i)e 



(t k -t k -l)r(C k _ 1 ) -(t-t k )r(C k ) 



n 

,n=l 



W(C„-i -> C„) 
r(C„_i) 



(6) 



At this stage, it may seem quite arbitrary. The justification for this point of view is mainly that it gives a coherent 
picture, and that the value of Z(s, t) and derived quantities can effectively be computed in several models. Actually, 
the idea was already implicitly present in a work by van Beijeren, Dorfman and Latz [3,4], in the case of a Lorentz 
gas with discs scatterers, and a hard sphere gas. 

In order to give a unified picture, it is useful to notice that Z(s,t) can be expressed in terms of an observable 



Q+ = E ln 



WjCn ->Cn+l) 

r(C n ) 



(7) 



Indeed one has Z(s, t) = (e~ s<5 + } and the dynamical partition function appears as the moment generating function 
of Q + . Then ip(s) is the corresponding cumulant generating function. The average sign (...) includes the average 
both on configurations and times. From now on, ip will be denoted by ip + (s) to recall that it is associated to the 
observable Q+. 

In practice, Z(s,t) is rarely evaluated directly from its definition (6). One rather evaluates directly tp(s), which 
turns out to be the largest eigenvalue of an operator W+(C,C') = W(C — > C) 1_s r(C') s — r{C)5c,c' which is a kind 
of evolution operator weighted according to the parameter s. In [6], we found that not only the eigenvalue but 
also the eigenvector associated with this operator can yield useful information on the dynamical structure of the 
system dynamics, in particular when dynamical transitions occur. 

The KS entropy can then be obtained from several definitions which can be shown to be equivalent: 
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h KS =- lim I (V P(C ,&,--- ,Cif)lnP(C ,Ci,--- ,Ck)) (8) 

=^;(o) (9) 

= -]im<^t> (10) 

The last relation was obtained by a similar derivation as the one performed for discrete time Markov processes 
by Gaspard [2]. 

In the spirit of Gaspard [7] , once the thermodynamic formalism is expressed in terms of an observable, one can 
generalize it to a whole family of observables 

K-1 

A(()=^a(C„,C„ + i) (12) 

n=0 

which depends only on the configurational part of the trajectory. Then V>a(s) = limt^oo | ln(e _sA ) is the moment 
generating function of A and can be obtained as the largest eigenvalue of an operator Wa defined in a way similar 
to W+. 

Among all these observables, at least two other ones than Q + seem interesting. The first one is K, i.e. the 
number of configuration changes within the time interval [0, t]. It is the simplest one one could think of, and still 
already contains some relevant information on dynamical phase transitions. 

The second one was introduced by Lebowitz and Spohn [8] 

^ W(C n+ i^C n ) 

It is this observable that verifies the fluctuation relation which, in terms of the moment generating function tps, 
reads tps(s) = i/)s(l — s) (we assume through the whole paper that the systems under consideration can take only 
a finite number of states). Moreover, in the steady state, Qs was identified as the integrated entropy flux [8] (we 
are now referring to the Boltzmann entropy S(t) — — J^c P{C,t) \xiP(C,t)). 

We have shown that, if one defines a twin observable Q- for the reversed trajectories, namely Q- = Yln=o m ^^("c^+Tf"^ : 
then the observable Q + that we have introduced in the frame of the thermodynamic formalism can be related to 
Qs via the relation 

Qs = Q+- Q- (14) 

and, similarly to what Gaspard did for discrete time systems [2], one can define a reversed KS entropy h§^ s = 
— limt^oo ^f-t- which relates, in the stationary state, to the entropy flux: 

<7f = — Oirr = hKS ~ ^KS (15) 

where /iks is defined by (8-11). 



3. Mean-Field models 



In general, calculating the cumulant generating function i\>a associated with the observable A is a difficult task 
(see [6,9] for examples). However, for mean-field models which can be described by a macroscopic order parameter 
(such as a magnetization), and for which a detailed balance relation is available, one can quite easily get the 
leading order of ipA in the limit of large systems, at least for A = K and A — Q + . 

The derivation is based on the fact that for a symmetric operator, the largest eigenvalue can be expressed as 

r {v\w yTa \v) 1 

A m ax = max | {nv) | 



(16) 
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3.1. Symmetrization of the operators Wk andW+ 



One can check, using the detailed balance relation 

w(r _ » r'\ 

(17) 



Peq(C') _ W(C -> C) 



p oq (c) w{c^cy 

that the following operators obtained by a similarity transformation from Wk and W+ are symmetric 



WT = T~ 1 W K T ; T(C,C) = pV\C) 6 c ,c> (18) 
w s +ym = T -i w+T . T(C,C) = Pe^C^rfC) - * Scci (19) 



As a property of similarity transformations, these symmetric operators have the same spectrum - and thus the 
same largest eigenvalue - as the original ones. 

To make the presentation easier to follow, we shall exemplify the remaining of the approach in the case of the 
Potts model. 

3.2. Definition of the mean-field Potts model 

In each site of a lattice of size N, a spin variable Oi can be in q different states. The mean-field Potts model is 
defined by the Hamiltonian 



i,j k — 1 

N, 



1 



9 £ 



where N k is the number of spins in state k and 9 k — the corresponding fraction. When a site switches from 
state n to state m, the corresponding energy variation writes 

AH = [N m -N n + 1] = -Jq [9 m - 6>„ + 1/N] (20) 
The model is endowed with a continuous time Glauber like dynamics with transition rates 

W(pi = n^aj =m)= e~ ^ (21) 

The equilibrium probability distribution 

PeviWi}) = ^e~ 0n (22) 

verifies detailed balance. However, in what follows, we shall rather describe the system at the level of the occupation 
numbers N = {N k } k=1 . Then the equilibrium distribution that obeys detailed balance reads 



^w-kSsr"" (23) 

with the transition rates 



W (N',N) = W (N N') = iV n e' 3 ^ [iVm - JV " +1, (24) 

where N' = {JVi,"- ,K = N n -l,--- , N' m = N m + 1, • • • ,JV,}. 

In the following, we shall consider the operator associated with the observable K: 

W X (N',N)=^ S N^N n -i5 N ^,N m+ in k ^ n , k ^ m 8 N , kiNk W (N ^N') -Ul =1 8 KtNk r(N) (25) 
where z = e _s . Using the same transformation as in (18), one obtains the symmetric operator 



(N',N) =zY, E II + V N ™] 

-iW*££< e ^ K ~ <+1] 

fc — 1 n m^n 



I I 1/2 
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For large N, and outside of the thermodynamic transition, one expects that the eigenvector associated with the 
largest eigenvalue will take the form 

V(N) ~ c JV/({efc> '==i-< ! ) 

Indeed, this is the case in particular for z = 1 (s = 0), i.e. for the equilibrium distribution with 



An expansion in the large N limit yields 
^W sym (N,N')Q(N') 



a B4^[N m -N n +i] Nf(e lt -,e q ) 



=*EE + m^e Nf(Bl '- ■ fl »+*-- -EE ^ 

= Ne Nf ^-^J2Yl {zVfijZe^-^ -e n e^ e ™- e ^} (27) 

The quantity to maximize is 

S N En> V(N)W^(N, N'MN') _ E " Ne™W"~ gn { WO^fc-*%r - fl w e^l« m -«»] } 

E N [^(N)] 2 ^ e 2N H e 1 ,-,e q) 

(28) 

For each function /, in the large N limit, the sum is dominated by the maximum of /, for which — Vn = l..q, 
i.e. 

= (E E [sVtt- e„e^' fl ™- fl »l] 1 (29) 
We shall now apply this result to various values of q. 
3.3. Arbitrary value of q 

As in [10], we assume a symmetry breaking along one particular direction k — 1 (assume for example that a 
very small field along this direction breaks the symmetry of the system). Then the system can be described by 
one order parameter m G [~l/(q - 1), 1] with 6 k = ±f± yfc > 1 and 0i = 1+(g ~ 1)m . 

The cumulant generating function for K is then obtained from 



max [ 2 *Zlzyfo - m) [1 + (q - l)m] + (g = 1)(g = 2) 
m I q q 



(z-l)(l-m) 



O—l PJq 0—1 P J<i • 

q [l + (g-l)m]e- — t(l- m ) e — m \ (30) 



iPk{s) 
N 

_g 

q " " ' ' q 

We shall now take special values of q to explore the consequences of this maximization condition. 
3.4. q = 2; the mean-field Ising model 

For q — 2, spins can take only two values Si = ±1. The corresponding fractions can be expressed in terms of a 
unique order parameter, the magnetization m: 

1 + m 1-m 
0i = — - — and 2 = — ^ — 

with di — 82 = m, and the Hamiltonian reads with these notations 



2iV ■ 

»,3 
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m m m 



Figure 2. The function that must be maximized in order to find the value of mftr(s) and — K N s , in the case of the Ising model 
(q— 2), for a temperature f3 — 1.5 under the critical temperature (ordered phase), J — 1, and for the s values —0.5, 0, and 0.1. 



Then the maximization condition (30) becomes 

t^l = max {zy/a - m 2 ) - cosh /3Jm + msmh/3Jm\ (32) 

The behavior of this function is illustrated in figures 1 and 2. The magnetization for which the maximum is 
obtained is called rriK(s). In the high temperature phase, it is vanishing for a whole range of s values. This was 
of course expected for s = 0, where niK coincides with the physical magnetization of the system. But above a 
certain threshold s c , the maximum splits into two symmetrical ones. The transition is continuous (see figure 3, 
left), and corresponds to a discontinuity in the second derivative of i/>x(s) (figure 4). A clear difference appears 
between the disordered and ordered phases. In the latter case {/3 > 1), the transition value s c is exactly at zero 
(figure 3, right). Besides, the transition in mx is now discontinuous, and the first derivative of i/>k(s) already 
shows a discontinuity. 

Once we know that the transition above T c is continuous, and thus that vhk [s) is close to zero at the transition, 
it is possible to determine the value for s c by expanding the expression for i/jk(s) for small txik values. Then the 
condition = yields a non-zero ma solution for 

z c = /3J(2- (3 J) (33) 

It is of course much harder to perform these calculations in finite dimension. It may nevertheless be argued 
that the dynamical phase transition occurring in the paramagnetic phase for s > s c > has an upper critical 
dimension d c = 6 characterized by critical exponents that are apparently unrelated to those of the static phase 
transition at (3 = 1 and s — 0. 
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s 



s 



Figure 3. The value of the magnetization for which the function of (32) is maximal, as a function of s, in the case of the Ising model 
(q=2), for = 0.5 (left) and = 1.5 (right). We took J = 1. 




S S 

Figure 4. tPk(s) in the case of the Ising model (q— 2), for — 0.5 (left) and — 1.5 (right). We took J = 1. The derivative Tp' K (s) 
is plotted in the inset. It is continuous (rcsp. discontinuous) in the disordered (ordered) phase. 

3.5. q = 3: the three states Potts model 



For q = 3, the critical temperature of the thermodynamic phase transition corresponds to f3 c = 2.7725 [10]. As 
we shall show graphically, the dynamical phase transition is now discontinuous both above and under the critical 
temperature 

For /3 = 0.5, figure 5 shows the behavior of the function to maximize. By contrast with the Ising case, now the 
transition in the ma parameter is discontinuous even in the high temperature phase (figure 6). Both for high and 
low temperatures, the discontinuity occurs in the first derivative of iPk{s), as seen in figure 7. However, while the 
transition at high temperature occurs at s — s c / 0, it occurs at s — under the critical temperature. 



3.6. The q — 1 limit 



The limit q = 1 is interesting because it can be related to percolation (see [11] for example). In order to have a 
well defined limit, one considers 

^k(s) = Km^zj =iVmax{^(2v / l _ ^ + m-l) + (1 - m) (l - c^ m ) - e~^ m } 

In the case 2 = 1, this expression can be factorized 

= max (vT^e^ - e^™} 2 (34) 
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s s 

Figure 6. The value of the magnetization for which the function of (32) is maximal as a function of s, in the case of the Potts model 
for q = 3, for = 0.5 (left) and = 5 (right). We took J = 1. 




S S 

Figure 7. i^k (s), in the case of the Potts model for q = 3, for — 0.5 (left) and — 5 (right). We took J — 1. The derivative ip' K (s) 
is plotted in the inset. It is discontinuous in both cases. 

As in the case q = 3, one finds that the dynamical phase transition that occurs at s — for /3 > 1 or < 1 is 
characterized by a discontinuous iuk (s), and a continuous cumulant generating function (the discontinuity occurs 
in the first derivative of ip K (s)). 



4. Conclusion 



Though the thermodynamic formalism was designed in order to study out-of-equilibrium systems, it is also a 
useful tool to shed a new light on equilibrium systems. Their dynamical properties turn out to be richer than could 
be expected. Actually, some dynamical phase transitions do occur even in apparently dull phases as the disordered 
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phase in the Ising model. Dynamical phase transitions and thermodynamic ones are connected : in all examples 
that we studied, the features of the dynamical phase transition are different on each side of the thermodynamic 
phase transition. The understanding of the interplay between dynamical and thermodynamic phase transitions 
could lead to a renewed vision of equilibrium systems. 

Here the case of the Potts model was studied for a various number of states q. Interestingly, the case q — 2 
turns out to be quite special in the sense that the paramagnetic phase of the Ising model is the only one where a 
continuous phase transition occurs in the order parameter 171k(s), corresponding to a discontinuity which appears 
only in the second derivative of ippc(s). By contrast, both for q — 1 or q — 3, all dynamical transitions were found 
to be discontinuous in the order parameter mif(s), and it is the first derivative of iPk{s) that shows a discontinuity. 

In a research field which often requires very technical and heavy calculations, the method that we present 
here is interestingly simple. We applied it in this paper to the calculation of the cumulant generating function of 
K for the mean-field Potts model in the large N limit, but more generally it can be applied to any mean-field 
model which can be described by a macroscopic order parameter, and for which a detailed balance relation at the 
macroscopic scale is available (in particular, the transition rates of the dynamics must be functions only of the 
aforementionned order parameter). Though all the results we presented were dealing with the observable K, the 
same kind of calculations - with the same kind of conclusions in the case of the Potts model - can be performed for 
the observable Q+ (the corresponding cumulant generating function is then the so-called topological pressure). It 
should be noticed however that in the case of Q+, the large N expansion of equation (27) is only valid for s < 1, 
as a consequence of 1 — s powers. A generalization to other observables of the form (12) with a(C,C') = a (N, N') 
is often possible. 
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